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(57) ABSTRACT 

A network appliance is configured to store and retrieve 
instructions for handling a packet corresponding to a con- 
nection. The network appliance includes a connection stor- 
ing processor configured to hash an incoming flow identifier 
for incoming packets to obtain an incoming hash; to hash an 
outgoing flow identifier for outgoing packets to obtain an 
outgoing hash; to insert an incoming entry corresponding to 
the incoming hash in a bidirectional hash table; and to insert 
an outgoing entry corresponding to the outgoing hash in a 
bidirectional hash table. A connection finding processor is 
configured to hash a packet identifier to obtain a packet hash; 
and to determine a matching entry in the bidirectional hash 
table that corresponds to the packet hash. A memory is 
configured to store the bidirectional hash table. 

40 Claims, 12 Drawing Sheets 



Start 




700 



Hash foreign and global IP addresses 



Copy pointer from hash table into 
connection object 



With pointer to inbound flag register of 
connection object in hash table 



Hash local and foreign IP addresses 



Copy pointer from hash table into 
connection object 



Write pointer to outbound flag register of 
connection object in hash table 



Available Copy 




End V- 714 



702 



704 



706 



708 



710 



712 



05/05/2004, EAST Version: 1,4.1 



U.S. Patent May 14, 2002 Sheet 1 of 12 US 6,389,419 



106 



102 

S 


104 


r - 




Server C 




Client 




Network 
appliance 




Server D 








Address A 
Port A 






Server E 



Address B 
PortB 



Figure 1A 



05/05/2004, EAST Version: 1.4.1 



U.S. Patent May 14,2002 Sheet 2 of 12 US 6,389,419 



104 



112 

_A_ 

Memory 



116 



114 



Outbound 
interface 



Processor 

S 

110 



Inbound 
interface 



Figure 1B 



05/05/2004, EAST version: 1.4.1 



U.S. Patent May 14, 2002 Sheet 3 of 12 US 6,389,419 Bl 





Foreign 


Local 


Global 


Inbound 


Source 




Destination 










Outbound 


Destination 


Source 





Figure 2A 



220 



Hash foreign IP address 



Use hash to index into hash table 



Copy pointer from hash table into 
new connection object 



Write pointer to new connection 
object into hash table 




End V- 230 



222 



224 



226 



228 



Figure 2B 



05/05/2004, EAST Version: 1.4.1 



U.S. Patent May 14, 2002 Sheet 4 of 12 US 6,389,419 Bl 



240 



Hash packet source IP address 



c 



Check hash list 



z 



Found 



Inbound packet 



x 



242 

244 
Not found 



Retrieve 
connection object 



246 



247 



Hash packet destination IP address 
and check hash list again 




254 



248 





Found \^ Not found 






No connection object exists 








\ 


Outbound packet 


^ 252 250 



Figure 2C 



05/05/2004, EAST Version: 1.4.1 



U.S. Patent May 14, 2002 Sheet 5 of 12 US 6,389,419 



268 272 274 276 




Hash 



Hash 



Hash 



Hash 



Hash 



Figure 2D 



266 



268 



Hash 



Hash 



Hash 



Hash 



Hash 



Hash 



274 

_1_ 



276 



CO 



CO 



Figure 2E 



05/05/2004, EAST Version: 1.4.1 



U.S. Patent May 14, 2002 Sheet 6 of 12 US 6,389,419 Bl 



302 



302a 



302b 



306 




310 



308 



Figure 3A 



Inbound / Outbound flag -~ 320 

Connection object pointer 322 



Next I/O object pointer 324 



Figure 3B 



05/05/2004, EAST Version: 1.4.1 



U.S. Patent May 14, 2002 Sheet 7 of 12 



US 6,389,419 Bl 



Foreign address 



Foreign port 



Local address 



Local port 



Global address 



Global port 



Actions / flags 



330 
332 
334 
336 
338 
340 
342 



Figure 3C 



05/05/2004, EAST Version: 1.4.1 



U.S. Patent May 14,2002 Sheet 8 of 12 US 6,389,419 



In = 0 



Next pointer 



In = 1 



Next pointer 



Connection 

object 
information 



400 
402 
404 
406 

408 



Figure 4A 



410 



f 



412 

1 





In = 0 
next 


. — > 


In = 1 
next 






416 



Figure 4B 



05/05/2004, EAST Version: 1.4.1 



U.S. Patent May 14, 2002 Sheet 9 of 12 



US 6,389,419 Bl 



500 



Hash foreign and global IP addresses 



Write pointer to connection object in 
inbound object 



Copy pointer from hash table into 
inbound object 



Write pointer to new inbound object into 
hash table 



Hash foreign and local IP addresses 



Write pointer to connection object in 
outbound object 



v 



Copy pointer from hash table into 
outbound object 



Write pointer to new outbound object into 
hash table 




End 



502 



504 



506 



508 



510 



512 



514 



516 



Figure 5 



05/05/2004, EAST Version: 1.4.1 



U.S. Patent May 14, 2002 Sheet 10 of 12 US 6,389,419 Bl 



Start 




600 



Hash packet source + destination 



c 



602 



Check hash table 



Not found 



604 



Found 



606 



No connection 
object exists 



Q Inbound flag set? 



608 



Inbound packet 



610 



Outbound packet 




612 



Figure 6 



05/05/2004, EAST Version: 1.4.1 



U.S. Patent May 14, 2002 Sheet 11 of 12 



US 6,389,419 Bl 



700 



Hash foreign and global IP addresses 



Copy pointer from hash table into 
connection object 



With pointer to inbound flag register of 
connection object in hash table 



Hash local and foreign IP addresses 



Copy pointer from hash table into 
connection object 



Write pointer to outbound flag register of 
connection object in hash table 




End 



714 



702 



704 



706 



708 



710 



712 



Figure 7 



05/05/2004, EAST Version: 1.4.1 



U.S. Patent 



May 14, 2002 



Sheet 12 of 12 



US 6,389,419 Bl 



Start 




800 



Hash packet source and destination 



c 



812 



Check hash table 



c 



Not found. 



Found 



Inbound flag set? 



Zff! 



Inbound 
packet 



Set memory offset 
to inbound value 



820 



822 



5 



814 



818 



816 



No connection 
object exists 



Not set 



Outbound 
packet 



Set memory offset 
to outbound value 




824 



826 



Figure 8 



05/05/2004, EAST Version: 1.4.1 



US 6,3! 

1 

STORING AND RETRIEVING CONNECTION 

INFORMATION USING BIDIRECTIONAL 
HASHING OF CONNECTION IDENTIFIERS 

CROSS REFERENCE TO RELATED 
APPLICATIONS 

This application is related to co-pending U.S. patent 
application Sen No. 08/850,248 entitled "Distributing Con- 
nections To A Group Of Machines" filed May 2,1997, which 
is incorporated herein by reference for all purposes. 

FIELD OF THE INVENTION 

The present invention relates generally to intercepting and 
processing packets that are related to a connection. More 
specifically, finding a connection object containing informa- 
tion related to the connection by looking once in a hash table 
that contains entries for packets sent in both directions 
between two parties is described. 

BACKGROUND OF THE INVENTION 

Various network appliances are used in networks to 
intercept packets and process packets. These appliances 
include load balancers, network address translation (NAT) 
devices, proxies, firewalls, and packet monitors. These 
devices monitor or modify packets on a network. In many 
cases, packets belonging to different connections are treated 
differently. Often, packets corresponding to different flows 
in the same connection are handled differently. 

Instructions for handling packets in different flows are 
stored by the network appliance for the purpose of deter- 
mining how to handle incoming packets. These instructions 
must be accessed quickly so that packet processing is not 
unduly delayed. FIG. 1A is a block diagram illustrating a 
network that includes a client 102, a network appliance 104, 
and a set of servers 106. For the purpose of this illustration, 
network appliance 104 will be described as a NAT device 
that translates the destination address of certain packets sent 
from the client to a global IP address assigned to the group 
of servers. Network appliance 104 translates the global IP 
address used by the client to the local IP address of one of 
the servers selected to handle the connection with the client. 
The client IP address and port are referred to as the foreign 
IP address and port. 

The network appliance modifies certain packets or records 
certain packets that belong to certain connections. A con- 
nection is made up of two flows, one in each direction 
between two parties. In general, a source address and port 
number, a destination address and port number, and a 
protocol define a flow. The source and destination addresses 
and ports are reversed for flows in opposite directions. For 
the purpose of illustration, this specification describes an 
example using only addresses as flow identifiers. It should 
be noted, whenever only an address is mentioned as an 
identifier, that an address and port may also be used and that 
a protocol may be added in some embodiments as well 

In the example shown in FIG. 1A, each connection is 
defined by a client IP address and port, a local IP address and 
port corresponding to the server selected to handle the 
connection, and a global IP address and port that corre- 
sponds to the IP address and port specified by the client for 
the connection. Packets passing through the network appli- 
ance are handled differently depending on whether they are 
a part of the flow from the client to the server or the flow 
from the server to the client. Therefore, network appliance 
104 must, for each packet received, find a connection object 
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that corresponds to the connection to which the packet 
belongs and also determine whether the packet is an incom- 
ing or an outgoing packet. For the purpose of this 
description, the terms incoming and outgoing are defined 

5 with reference to the server. 

FIG. IB is a block diagram illustrating the components of 
a typical network appliance. A processor 110 is connected to 
a memory 112 and several inbound/outbound interfaces that 
allow the network appliance to send and receive packets. 

30 FIG. IB shows an inbound interface 114 and an outbound 
interface 116. Each of the interfaces are intended to repre- 
sent a large number of interfaces. In some embodiments, the 
inbound/outbound interfaces are distinguished by whether 
they are on the client side or the server side. The network 

15 appliance may be implemented on any suitable general 
purpose computer architecture, including a machine running 
UNIX or Microsoft Windows. 

FIG. 2A is a chart illustrating which address is included 
as the source and destination address for inbound and 

20 outbound packets. For an inbound packet, the foreign 
address is the source address and the global address is the 
destination address. For an outbound packet, the foreign 
address is the destination address and the local address is the 
source address. The foreign address is included in both 

25 inbound and outbound packets as either the source address 
or the destination address. As a result, a connection object 
that corresponds to a packet can be located by hashing only 
the foreign address included in the connection object. A 
match in the hash table that corresponds to the connection 

30 object can be located by first searching using the source 
address of the packet and then searching using the destina- 
tion address of the packet. If the connection object is found 
using the source address of the packet, then the packet is an 
inbound packet that includes the foreign address of the 

35 connection as its source address. If the search using the 
packet destination address locates the connection object, 
then the packet is an outbound packet and the foreign 
address of the connection is the destination address of the 
outbound packet. Thus, two lookups are required at most to 

40 locate a hash table entry that corresponds to the correct 
connection object. 

FIG. 2B is a flowchart illustrating a process for creating 
a new entry in a hash table for a new connection object. The 

45 process starts at 220. In a step 222, the foreign IP address of 
the connection objects is hashed. Next, in a step 224, the 
hash is used to index into the hash table. In a step 226, the 
pointer from the hash table is copied into the new connection 
object. Then, in a step 228, a pointer to the new connection 

5Q object is written into the hash table. The process ends at 230. 
FIG. 2C is a flow chart illustrating a process for searching 
for a connection object when a packet is received. The 
process starts at 240. In a step 242, the packet source IP 
address is hashed. Then, in a step 244, the hash list is 

55 checked to see if the source IP address hash is included in 
the list. If the source IP address hash is found, then in step 
246, it is determined that the packet is an inbound packet. In 
a step 247, the connection object is retrieved and the process 
ends at 254. 

60 If, in step 244, the source IP address hash is not found, 
then control is transferred to a step 248. In step 248, the 
packet destination IP address is hashed and the hash list is 
checked again. If the destination IP address hash is not 
found, then control is transferred to a step 250 and it is 

65 determined that no connection object exists. If the destina- 
tion IP address hash is found, then control is transferred to 
a step 252 and it is determined that the packet is an outbound 
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packet. Control is then transferred to step 247 and the locations that contain information about whether the packet 

connection object is retrieved. Thus, the connection object is matching the hash table entry is inbound or outbound. The 

retrieved and it is determined whether the packet is an memory location may be part of a connection object or may 

inbound packet or an outbound packet. point to a connection object. Thus, the connection object 

FIG. 2D is a block diagram illustrating a hash table and s corresponding to a packet may be found by hashing the 

i_* . . • , , , , source and destination addresses of the packet and doing a 

several connection objecte which may be searched using the ^ looku . ^ hash uble „,„ hashes in 5 * h 

hash Uble. Hash table 266 includes a hash 26* Hash 268 ^tclion, for each connection object. 

mcludes a pointer to a connection object 272. Connection , , . . . . , 

• . t ^ -i-j . i. • * * It should be appreciated that the present invention can be 

obiect 272 points to connection object 274, which points to . t . . rr . r . 

, - , ™ - „ 4 if- m implemented m numerous ways, including as a process, an 

connection obiect 276. The pointer to connection object 272 10 v ^ . . J ' .5 

. • . . . u 1 . apparatus, a system, a device, a method, or a computer 

was entered in the hash table using a process such as is rr , . . ,/ . , . . / 

, _ . . ™_ c .J .. readable medium such as a computer readable storage 

desenbed m FIG. 2B. The foreign IP address in connection ,. t * . . 4 & 

. . „ *• i_- * <*ia *>ii*u u medium or a computer network whe rem program mstruc- 

obiect 272, as well as connection objects 274 and 276 bashes A . . , , , - • 

.1 l *Jo i_* * if* • *u 1 i „• lions are sent over optical or electronic communication 

to hash 268. Connection object 276 is the last connection v . « 1 ■ *• u j- * * *l 

....... * , • n • , m links. Several mventive embodiments of the present lnven- 

obiect in the hash chain and so it contains a null pointer. 15 , , , , . r 

J r tion are described below. 

FIG. 2E is a block diagram iUustratmg hash table 266 just fa 0Qe embodiment) a method of storin ^ retrievi 
before connection object 272 is inserted in the hash table. mstructions for handling a packct corresponding to a con- 
Hash 268 mcludes a pointer labeled A that points to ^ { ^ incoming flow identifier for 
connection object 274. Connection object 274 includes a in^ng p ackets to obtain an incoming hash and hashing an 
pointer labeled «B" that points to connection object 276. . flow {qt fa ketg tQ Qbtain an 
Connection object 272 Sis inserted by copying pointer A into . faash M incomi ent corresponding to the 
connection object 272 and then overwriting pointer A in mcomirjg hasn fa mserted m a b i direc tional hash table and an 
hash table 266 with a pointer to connection object 272. outgoing en try corresponding to the outgoing hash is 
Pointer B in connection object 274 remains unchanged inserted in the bidirectional hash table. A packet identifier is 
Thus, connection object 272 is inserted at the beginning of hashed lQ oblain a packet faash ^ a matching entry m the 
the hash chain associated with hash 268. bidirectional hash table is determined that corresponds to the 

When addresses are described as being hashed and used packet hash, 

to find a connection object, more than one connection object [n QQe embodim e n t, a network appliance configured to 

may correspond to a set of addresses because different ports 3Q store and retrieve instructions for handling a packet corre- 

may be used for different connections between those S ponding to a connection includes a connection storing 

addresses. The one connection object that matches a packet pr0C essor configured to hash an incoming flow identifier for 

can be determined by examining the port numbers in a set of incoming packets to obtain an incoming hash; to hash an 

connection objects identified using the hash table. out going flow identifier for outgoing packets to obtain an 

Alternatively, port numbers can be included in identifiers 35 outgo i ng hash; to mser t an incoming entry corresponding to 

before hashing so that connection objects with different mc incoming hash in a bidirectional hash table; and to insert 

ports are not retrieved together. In general, the term flow ^ outgoing entry corresponding to the outgoing hash in a 

identifier is used to describe any information in a connection bidirectional hash table. A connection finding processor is 

object used to find the flows belonging to the connection configured to hash a packet identifier to obtain a packet hash; 

object. The term packet identifier is used to describe any ^ ^ tQ dctcrmme a matching entry in the bidirectional hash 

information in a packet header that is used to find the table ^ corresponds to the packet hash. A memory is 

connection or flow corresponding to the packet. In the configured to store the bidirectional hash table, 

example described herein, the identifiers are combinations {q oqc embodiment) a computcr readable medium 

of IP addresses. includes program code for storing and retrieving information 

It would be useful if the lookup process for connection 45 f or handling a packet corresponding to a connection. The 
objects could be simplified and sped up for incoming program code comprising instructions for hashing an incom- 
packets. Since a very large number of packets is likely to be fj ow identifier for incoming packets to obtain an incom- 
received for a given connection, the resources of the network mg nasn and hashing an outgoing flow identifier for outgo- 
appliance may be taxed as the lookup process is executed mg pac kets to obtain an outgoing hash. An incoming entry 
many times. Maximizing the efficiency of the lookup pro- 50 corresponding to the incoming hash is inserted in a bidirec- 
cess would result in reduced latency in the network appli- tional hash table and an outgoing entry corresponding to the 
ance. outgoing hash is inserted in the bidirectional hash table. A 

Pir ,... Anv „ ™j C Ivnn:wTinw packet identifier is hashed to oblain a packet hash and a 

SUMMARY OF THE INVENTION ^ &tching entfy m ^ bidirectional hash ^ is determm ed 

Accordingly, a method is described for storing a connec- 55 that corresponds to the packet hash, 

tion object using a hash table and then finding the connection These and other features and advantages of the present 

object that corresponds to a packet in a flow with a single invention will be presented in more detail in the following 

lookup. Each connection object is hashed twice — once in detailed description and the accompanying figures which 

each direction relative to a virtual server. For inbound illustrate by way of example the principles of the invention, 

packets toe foreign and global ad^s are hashed sin« 60 DESCRIPTION OF THE DRAWINGS 
those addresses correspond to the source and destination 

addresses of incoming packets. For outbound packets, the The present invention will be readily understood by the 

local and foreign addresses arc hashed, since those addresses following detailed description in conjunction with the 

correspond to the source and destination addresses of out- accompanying drawings, wherein like reference numerals 

going packets. Two pointers, one for each hash, are entered 65 designate like structural elements, and in which: 

in a hash table used to search for a connection object. The FIG. 1A is a block diagram illustrating a network that 

inbound and outbound hash table entries point to memory includes a client, a network appliance, and a set of servers. 
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FIG. IB is a block diagram illustrating the components of (corresponding to an incoming packet.) Then, the local IP 

a typical network appliance. address and foreign IP address are hashed (corresponding to 

FIG. 2A is a chart illustrating which address is included an outgoing packet.) 

as the source and destination address for inbound and As a result of this bi-directional hashing, when a packet 

outbound packets. 5 ^ rccc i ve d» ^ source and destination addresses of the 

m . a L • r packet may be hashed and a match in the hash table occurs 

FIG. 2B is a flowchart ulustrating a process for creating * r ci(hcr ^ incomin hash Qr {hc { hash (cd 

an new entry in a hash table for a new connection object. for ^ connection objecl . In u, c example haSQ 302 a 

FIG. 2C is a flow chart illustrating a process for searching corresponds to an inbound packet. Hash 302A includes a 

for a connection object when a packet is received. 1Q pointer to an inbound/outbound object 304 with its inbound/ 

FIG. 2D is a block diagram illustrating a hash table and outbound flag set, indicating that the hash table entry cor- 

several connection objects which may be searched using the responds to an inbound packet. Inbound/outbound object 

hash table. 304 contains a pointer to a connection object 310 that 

FIG 2E is a block diagram illustrating hash table 266 just corresponds to hash 3Q2A. Inbound/outbound object 304 

before connection object 272 is inserted in the hash table. 15 ^° ' mchidcs a P 0 *^ t0 next inbound/outbound object 

- A . . ... , . t 306 on its hash chain. Inbound/outbound object 306 includes 

FIG. 3A is a block diagram illustrating a memory struc- tl . . . . . . . J ., . . 

, .,. j- l.li a null pointer since it is the last inbound/outbound object on 

ture that includes a bi-directional hash table. . t , r , , . „ , afKlT , , . , . J r 

its hash chain. Hash 302B corresponds to the hash of the 

FIG. 3B is a block diagram illustrating the fields con- local Ip address and foreign IP address of an outbound 

tained in an inbound/outbound object. ^ pac k et . Hash 302B includes a pointer to an inbound/ 

FIG. 3C is a block diagram illustrating the structure of a outbound object 308 with its inbound/outbound flag not set. 

connection object. Inbound/outbound object 308 includes a pointer to connec- 

FIG. 4A is a block diagram illustrating a connection tion object 310. Inbound/outbound object 308 also contains 

object used in an embodiment that does not require inbound/ a null pointer since it is the last inbound/outbound object on 

outbound objects. 25 ^ nash cnanl * 

FIG. 4B is a block diagram illustrating an alternate Thus, inbound and outbound packets can be identified and 

memory structure that includes a bi-directional hash table. connection object 310 can be accessed using a single lookup 

FIG. 5 is a flowchart illustrating a process for entering a m table 302 " H ^^* l™ 1 ? f ° f 

new connection object in a hash table that uses inbound/ and ^t Points to an inbound/ou bound object that specifies 

outbound ob'ects 30 connection object and the fact that the packet is an 

J , t inbound packet. Likewise, bash 302B is found for an out- 

FIG. 6 is a flowchart illustrating a process for finding a bound ket Hash 302B ints tQ an htouayauibovmd 

connection object using the memory structure illustrated in objcct ^ mdicatcs ^ tfac packct is M outbourld packet 

FIG. 3A. and jjgQ points to the connection object for the packet. An 

FIG. 7 is a flowchart illustrating a process for storing a 35 alternative hash table and connection object structure that 

new connection object using the memory structure illus- does not use inbound/outbound objects is described in FIG. 

trated in FIG. 4B. 4B. 

FIG. 8 is a flowchart illustrating a process for finding a FIG. 3B is a block diagram illustrating the fields con- 
connection object in the memory structure illustrated in FIG. tained in an inbound/outbound object. An inbound/outbound 
4B when a packet is received. 40 flag 320 is set or not set for the purpose of indicating whether 

nccr^DiDTrnM * ne P ac ^ et * s an inbound or an outbound packet. A connec- 

DETAILED DESCR1P HON ^ ob j ect pointer 322 points to a connection object. A next 

A detailed description of a preferred embodiment of the inbound/outbound object pointer 324 points to the next 

invention is provided below. While the invention is inbound/outbound object in the hash chain. The inbound/ 

described in conjunction with that preferred embodiment, it 45 outbound object may also include other fields that contain 

should be understood that the invention is not limited to any additional information, if desired. 

one embodiment. On the contrary, the scope of the invention FIG. 3C is a block diagram illustrating the structure of a 

is limited only by the appended claims and the invention connection object. The connection object includes a foreign 

encompasses numerous alternatives, modifications and address 330, a foreign port 332, a local address 334, a local 

equivalents. For the purpose of example, numerous specific 50 port 336, a global address 338, and a global port 340. In 

details are set forth in the following description in order to addition, the connection object may also include other fields 

provide a thorough understanding of the present invention. 342 that specify various actions to be performed on packets 

The present invention may be practiced according to the as well as various flags indicating the status of the connec- 

claims without some or all of these specific details. For the tion or other information about the connection object. The 

purpose of clarity, technical material that is known in the 55 connection object is used by the network appliance to 

technical fields related to the invention has not been determine what to do with the packets that it receives, 

described in detail in order not to unnecessarily obscure the Finding the connection object quickly is important to the 

present invention. performance of the network appliance. Finding the connec- 

FIG. 3A is a block diagram illustrating a memory struc- tion object with only a single hash table lookup using the 

ture that includes a bi-directional hash table 302. 60 structure shown in FIG. 3A greatly increases the efficiency 

Bi-directional hash table 302 allows the hash chain includ- of the connection object lookup process. This efficiency 

ing the connection object that matches an incoming packet comes at the cost of requiring two hashes to be performed to 

to be located after only a single hash table lookup. add a connection object to the hash table (as well as two 

Bi-directional hash table 302 is generated by hashing not deletions when a connection object is deleted form the hash 

only the foreign IP address of each connection object. 65 table). Also, both the source and destination addresses of 

Instead, each connection object is hashed twice. First, the incoming packets must be hashed before the hash table is 

foreign IP address and global IP address are hashed searched. These additional requirements, however, are out- 
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weighed by the benefit of avoiding two hash table lookups 
for finding certain connection objects. 

FIG. 4 A is a block diagram illustrating a connection 
object used in an embodiment that does not require inbound/ 
outbound objects. Memory location 400 stores an inbound/ 
outbound flag with a zero value. Any pointer that points to 
memory location 400 will point to the zero flag. The zero 
flag indicates that the packet that generated a match in the 
bi-directional hash table is an outbound packet. Memory 
location 402 is a pointer to the next connection object in the 
hash chain. Inbound/outbound flag 404 is in a different 
memory location than inbound/outbound flag 400. A hash 
table entry that points to inbound/outbound flag 404, which 
is set, corresponds to an inbound packet. The set flag 
indicates that the packet that generated the match with the 
hash table is an inbound packet. A pointer 406 to the next 
connection object in the hash chain is also included. Finally, 
connection object information 408 contains the information 
included in the connection object for handling packets. 

FIG. 4B is a block diagram illustrating a memory struc- 
ture that includes a bi-directional hash table 410. The use of 
the inbound/outbound objects is eliminated by including 
inbound/outbound flags in each connection object. Connec- 
tion object 412 and connection object 416 are both part of a 
hash chain that includes connection object 414. Each con- 
nection object essentially acts as two different connection 
objects, one corresponding to an inbound flow and the other 
corresponding to an outbound flow. 

When a hash that matches one of the hashes in the hash 
table is generated by the inbound flow identifier of the 
connection object, a pointer to the memory location in the 
connection object where the inbound/outbound flag is set is 
generated. Likewise, when the outbound flow identifier of a 
connection object is hashed, a pointer to the flag in the 
connection object that indicates an outbound flow is gener- 
ated. 

When the connection object is read, the memory location 
pointed to by the hash table or by the previous connection 
object in the hash chain immediately indicates whether the 
packet is inbound or outbound. The remainder of the infor- 
mation in the connection object may be accessed by offset- 
ting the memory by an amount that corresponds to either the 
difference between the memory address of the inbound/ 
outbound flag that is not set and the desired connection 
object element or the difference between the memory 
address of the inbound/outbound flag that is set and the 
memory address of the connection object element. By read- 
ing the inbound/outbound flag, the proper offset can be 
determined. Thus, without requiring an inbound/outbound 
object, the modified connection object shown enables a 
determination of whether a new packet is an inbound or an 
outbound packet. 

FIG. 5 is a flowchart illustrating a process for entering a 
new connection object in a hash table that uses inbound/ 
outbound objects. The process starts at 500. At step 502, the 
foreign and global IP addresses of the connection object are 
hashed. Next, in a step 504, a pointer is written to the 
connection object in an inbound object. It should be noted 
that when the inbound/outbound flag is set, an inbound/ 
outbound object is referred to as an inbound object and when 
the inbound/outbound flag is not set, an inbound/outbound 
object is be referred to as an outbound object. 

In a step 506, the pointer from the hash table matching the 
foreign and global IP address hash is copied into the inbound 
object. In a step 508, a pointer to the new inbound object is 
copied into the hash table. After step 508, the hash table 
entry for inbound objects has been made. 
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In a step 510, the foreign and local IP addresses of the 
connection object are hashed. Next, in a step 512, a pointer 
to the connection object is inserted into an outbound object. 
In a step 514, a pointer from the hash table is copied into the 

5 outbound object. In a step 516, a pointer to the new 
outbound object is written into the hash table. The process 
then ends. Thus, two entries are made in the bi-directional 
hash table, one corresponding to each of the flows in 
different directions that make up the connection object. 

10 FIG. 6 is a flowchart illustrating a process for finding a 
connection object using the memory structure illustrated in 
FIG. 3A The process starts a 600. In a step 602, the packet 
source and destination addresses are hashed. Next, in a step 
604, the hash table is checked. The match looked for in the 

15 hash table depends on whether the pointer in the hash table 
points to an inbound/outbound flag that is set If the inbound/ 
outbound flag is set, then the connection object foreign and 
global addresses are compared to the packet source and 
destination addresses, respectively. If the inbound/outbound 

20 flag is not set, then the connection object local and foreign 
addresses are compared to the packet source and destination 
addresses, respectively. If a match to the packet source and 
destination address is not found, then control is transferred 
to a step 606 and it is determined that no connection object 

25 exists. If the hash is found, then a step 608 branches to either 
step 610 or 612 depending on whether the pointer in the hash 
table points to an inbound/outbound flag that is set. If the 
inbound/outbound flag is set, then it is determined in a step 
610 that the packet is an inbound packet and the process 

30 ends at 614. If the inbound/outbound flag is not set in step 
608, then control is transferred to a step 612 and it is 
determined that the packet is an outbound packet. The 
process then ends at 614. 

Thus, a newly received packet is not hashed twice with 

35 each hash checked versus the hash table. Instead, both the 
source and the destination address of the packet are hashed 
and a single hash table lookup is performed to find the entry 
that corresponds to the connection object for the packet. Two 
separate hash table look ups, one for the source address and 

40 one for the destination address are not necessary. It is 
determined whether the packet is an inbound packet or an 
outbound packet by checking the value stored at the memory 
location inside the inbound/outbound object that the pointer 
in the hash table designates. Connection objects are found 

45 after at most one hash table look up instead after at most two 
lookups. 

FIG. 7 is a flowchart illustrating a process for storing a 
new connection object using the memory structure illus- 
trated in FIG. 4B. The process starts at 700. In a step 702, 

50 the foreign and global IP addresses of the new connection 
object are hashed. Next, in a step 704, a pointer from the 
hash table is copied into the new connection object. In a step 
706, a pointer to the inbound/outbound flag register of the 
connection object is written into the hash table. In a step 708, 

55 the local and foreign IP addresses of the connection object 
are hashed. The pointer is copied from the hash table into the 
connection object in a step 710. A pointer to the outbound 
flag register of the connection object is written into the hash 
table in a step 712. The process ends at 714. 

60 Thus, the connection object includes both an inbound/ 
outbound flag register and an outbound flag register. The 
inbound/outbound flag register indicates that an inbound 
packet has generated a match and the outbound register 
indicates that an outbound packet has generated a match. A 

65 matching hash points to either an inbound register or an 
outbound register and when the value of the register is read, 
it can be determined whether the matching packet is inbound 
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or outbound. Inbound and outbound packets are discrimi- ing both the process and apparatus of the present invention, 

nated without requiring another object such as inbound/ For example, the bi-directional hash table described herein 

outbound object to be defined. The memory location that is may be used to store and retrieve connection objects for a 

pointed to by the hash list indicates the type of packet that NAT device that also translates the foreign IP address and 

generated the match. 5 port of incoming packets before forwarding such packets to 

FIG. 8 is a flowchart illustrating a process for finding a one of the servers. In such a case, the foreign address and the 

connection object in the memory structure illustrated in FIG. global address could be used to generate the incoming hash 

4B when a packet is received. The process starts at 800. In and the local address and the address that replaces the 

a step 812, the packet source and destination address are foreign address could be used to generate the outgoing hash, 

hashed. Then, in a step 814, the hash table is checked. The 10 Accordingly, the present embodiments are to be considered 

match looked for in the hash table depends on whether the »s illustrative and not restrictive, and the invention is not to 

pointer in the hash table points to an inbound/outbound flag be umited 10 me details P*™ herein » bm ma y be modified 

that is set. If the inbound/outbound flag is set, then the ™ thi n *** scope and equivalents of the appended claims, 

connection object foreign and global addresses are com- 1 15 c | ajmed ls: . . . . 

pared to the packet source and destination addresses, respec- 35 . ^ A method of storing and retrieving instructions for 

lively. If the inbound/outbound flag is not set, then the 35 a P ackel ™™I™ding * a connection compns- 

connection object local and foreign addresses are compared mS hasnin an flow idenlifier for kels 

to the packet source and destination addresses, respectively. P . . & . , , & r 

F . , m * . , , r . \ to obtain an incoming hash; 

If a match to the packet source and destination address is not . , . t . a . , , . . t 

found, then control is transferred to a step 816 and it is 20 ««gP»»g ^identifier for outgoing packets 

. ■ * rr i * • to obtain an outgoing hash; 

determined that no connection object exists. If an element is • ^- - • j- * • 

- JiLi iL t . i, uu inserting an incoming entry corresponding to the mcom- 

found that matches the packet source and destination hash, . . to . • i u u . ui 

, r t o^o j l ing hash in a bidirectional hash table; 

then control is transferred to a step 818 and the memory * 4 . t t t , t . 

, ... . iL t_ . * l i • * * lnserUng an outgoing entry corresponding to the outgoing 

locationm the ^connection object that the hash table points to hash in the bidirectional hash table; 

is checked to determine whether the mbound/outbound flag „ . t . , , , , , 

Tr A . „ . 4 iL . i . , c j . * " hashing a packet identifier to obtain a packet hash; and 

is set. If the flag is set, then control is transferred to a step ° . r . *. « , « 

ma j • j , - j . u . *u i™» v ■ determining a matching entry in the bidirectional hash 

820 and it is determined that the new packet is an inbound , t , * j p ' . , , 

. , , 0 -- jr * * 4 , table that corresponds to the packet hash, 

packet Then, in a step 822 a memory offset is set to the 2 A ^ m ^ x wherein me flow 

inbound value. The process then ends at 828. If, in step 818, identifiers contain IP addresse s. 

it is determined that the inbound/outbound flag is not set, 30 3 A method ^ recited ^ claim j wherein the flow 

then it is determined that the packet is an outbound packet identifiers contain IP addresses and port numbers, 

in step 824 and the memory offset is set to the outbound 4 A mct h od as recited in claim 1 wherein the incoming 

value in step 826. The process ends at 828. flow identifier includes a foreign address and a global 

Thus, when a new packet is received and a match is found address, 

in the hash table for the packet source and destination hash, 35 5. a method as recited in claim 1 wherein the incoming 

the pointer in the hash table is followed to a memory fl ow identifier includes a foreign address, a foreign port, a 

location in the connection object. That memory location is global address and a global port. 

read to determine whether an inbound/outbound flag is set or 5, a method as recited in claim 1 wherein the incoming 

not. The state of the flag determines whether the packet that fl ow identifier includes a foreign address, a foreign port, a 

generated the match for the pointer is an inbound or out- A0 global address a global port and a protocol, 

bound packet. The memory structure of the connection 7 a method as recited in claim 1 wherein the outgoing 

object includes two different flags. The memory offset flow ideDtifier includes a local address and a foreign address, 

between each of the flags and the other data in the connec- g t a method as recited in claim 1 wherein the outgoing 

tion object differs slightly as a result of the flags being flow identifier includes a local address, a local port, a foreign 

located in slightly different places in the structure of the 45 address and a foreign port. 

connection object. Therefore, the status of the inbound/ 9. a method as recited in claim 1 wherein the outgoing 

outbound flag is used to determine a memory offset so that fl ow identifier includes a local address, a local port, a foreign 

the rest of the data in the connection object may be read. address, a foreign port and a protocol. 

A bi-directional hashing technique for storing and retriev- 10. A method as recited in claim 1 wherein the matching 

ing connection objects using a bi-directional hash table has 50 entry points to a memory location that indicates whether the 

been described. Hash values are generated in the hash table matching entry was inserted as an incoming entry or an 

for both inbound and outbound packets. The source and outgoing entry. 

destination addresses of newly arrived packets are hashed U. A method as recited in claim 10 wherein the memory 

and the inbound or outbound status is determined by the location is part of an inbound/outbound object that points to 

element in the hash table that generates a match. A pointer 55 a connection object. 

points to a memory location either in a connection object or 12. A method as recited in claim 11 further including 

an inbound/outbound object that indicates the inbound or determining a memory offset based on whether the memory 

outbound status of the newly arrived packet. If an inbound/ location indicates that the matching entry was inserted as an 

outbound packet is used, a pointer to the connection object incoming entry or an outgoing entry for the purpose of 

is included in the inbound/outbound object. Thus, the con- 60 reading data from the connection object, 

nection object containing instructions for handling the newly 13. A method as recited in claim 10 wherein the memory 

arrived packet may be found. location is part of a connection object. 

Although the foregoing invention has been described in 14. A computer readable medium as recited in claim 1 

some detail for purposes of clarity of understanding, it will wherein the flow identifiers contain IP addresses, 

be apparent that certain changes and modifications may be 65 IS. A computer readable medium as recited in claim 1 

practiced within the scope of the appended claims. It should wherein the flow identifiers contain IP addresses and port 

be noted that there are many alternative ways of implement- numbers. 
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16. A computer readable medium as recited in claim 1 
wherein the incoming flow identifier includes a foreign 
address and a global address. 

17. A computer readable medium as recited in claim 1 
wherein the incoming flow identifier includes a foreign s 
address, a foreign port, a global address and a global port. 

18. A computer readable medium as recited in claim 1 
wherein the incoming flow identifier includes a foreign 
address, a foreign port, a global address a global port and a 
protocol. to 

19. A computer readable medium as recited in claim 1 
wherein the outgoing flow identifier includes a local address 
and a foreign address. 

20. A computer readable medium as recited in claim 1 
wherein the outgoing flow identifier includes a local address, 15 
a local port, a foreign address and a foreign port 

21. A computer readable medium as recited in claim 1 
wherein the outgoing flow identifier includes a local address, 
a local port, a foreign address a foreign port and a protocol. 

22. A computer readable medium as recited in claim 1 20 
wherein the matching entry points to a memory location that 
indicates whether the matching entry was inserted as an 
incoming entry or an outgoing entry. 

23. A computer readable medium as recited in claim 22 



28. A network appliance as recited in claim 26 wherein the 
flow identifiers contain IP addresses. 

29. A network appliance as recited in claim 26 wherein the 
flow identifiers contain IP addresses and port numbers. 

30. A network appliance as recited in claim 26 wherein the 
incoming flow identifier includes a foreign address and a 
global address. 

31. A network appliance as recited in claim 26 wherein the 
incoming flow identifier includes a foreign address, a for- 
eign port, a global address and a global port. 

32. A network appliance as recited in claim 26 wherein the 
incoming flow identifier includes a foreign address, a for- 
eign port, a global address a global port and a protocol. 

33. Anetwork appliance as recited in claim 26 wherein the 
outgoing flow identifier includes a local address and a 
foreign address. 

34. Anetwork appliance as recited in claim 26 wherein the 
outgoing flow identifier includes a local address, a local port, 
a foreign address and a foreign port. 

35. Anetwork appliance as recited in claim 26 wherein the 
outgoing flow identifier includes a local address, a local port, 
a foreign address a foreign port and a protocol. 

36. Anetwork appliance as recited in claim 26 wherein the 
matching entry points to a memory location that indicates 



wherein the memory location is part of an inbound/outbound is whcther ^ matc hing entry was inserted as an incoming 
object that points to a connection object. 

24. A computer readable medium as recited in claim 22 
wherein the memory location is part of a connection object. 

25. A computer readable medium as recited in claim 22 
further including determining a memory offset based on 30 
whether the memory location indicates that the matching 
entry was inserted as an incoming entry or an outgoing entry 
for the purpose of reading data from the connection object. 

26. Anetwork appliance configured to store and retrieve 



entry or an outgoing entry. 

37. Anetwork appliance as recited in claim 36 wherein the 
memory location is part of an inbound/outbound object that 
points to a connection object. 

38. Anetwork appliance as recited in claim 36 wherein the 
memory location is part of a connection object. 

39. A network appliance as recited in claim 38 further 
including determining a memory offset based on whether the 
memory location indicates that the matching entry was 



instructions for handling a packet corresponding to a con- 35 ^ cn&d as ^ mcornirig entry or an outgoing entry for the 



nection comprising: 

a connection storing processor configured to: 

hash an incoming flow identifier for incoming packets 

to obtain an incoming hash; 
hash an outgoing flow identifier for outgoing packets to 

obtain an outgoing hash; 
insert an incoming entry corresponding to the incoming 

hash in a bi-directional hash table; and 
insert an outgoing entry corresponding to the outgoing 
hash in a bi-directional hash table; 
a connection finding processor configured to: 

hash a packet identifier to obtain a packet hash; and 
determine a matching entry in the bi-directional hash 
table that corresponds to the packet hash; and 
a memory configured to store the bi-directional hash 
table. 

27. Anetwork appliance as recited in claim 26 wherein the 
connection storing processor and the connection finding 
processor are implemented on the same processor. 



40 



45 



50 



purpose of reading data from the connection object. 

40. A computer readable medium including program code 
for storing and retrieving information for handling a packet 
corresponding to a connection, the program code comprising 
instructions for. 

hashing an incoming flow identifier for incoming packets 

to obtain an incoming hash; 
hashing an outgoing flow identifier for outgoing packets 

to obtain an outgoing hash; 
inserting an incoming entry corresponding to the incom- 
ing hash in a bi-directional hash table; 
inserting an outgoing entry corresponding to the outgoing 

hash in a bi-directional hash table; 
hashing a packet identifier to obtain a packet hash; and 
determining a matching entry in the bidirectional hash 
table that corresponds to the packet hash. 
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